"Bag of keypoints"-based biomedical image search with affine covariant region detection and correlation-enhanced similarity matching

This paper presents a “bag of keypoints” based biomedical image retrieval approach by detecting affine covariant regions. The covariant regions simply refers to a set of pixels or interest points which are invariant to affine transformations, as well as occlusion, lighting and intra-class variations. To describe the intensity pattern within the interest points the Scale-Invariant Feature Transform (SIFT) is used. The SIFT features are then vector quantized to build a visual vocabulary of keypoints by utilizing the SelfOrganizing Map (SOM)-based clustering. By mapping the interest points extracted from one image to the words in the visual vocabulary, their occurrences are counted and the resulting histogram is called the “bag of keypoints” for that image similar to the “bag of words” based representation of documents in text retrieval. To exploit the correlations between the keypoints in the collection, a global similarity matrix is constructed to be utilized in a distance measure function to compare the query and database images. A systematic evaluation of image retrieval on a biomedical image collection demonstrates the advantages of the proposed image representation and similarity matching approaches in terms of precision-recall.